Model: allenai/OLMo-7B-0424-hf

Layer: 31. Neuron Index: 9634

gate+_in+

gate+_in-

gate-_in+

gate-_in-
Frequency: 14.67%. Frequency: 67.70%. Frequency: 0.29%. Frequency: 17.34%.
hook_post:
Max: 3.48;
Min: 0.00;
Avg: 0.16. 		hook_post:
Max: 0.00;
Min: -5.12;
Avg: -0.18. 		hook_post:
Max: 0.00;
Min: -0.49;
Avg: -0.08. 		hook_post:
Max: 1.58;
Min: 0.00;
Avg: 0.20.
hook_pre_linear:
Max: 3.11;
Min: 0.00;
Avg: 0.25. 		hook_pre_linear:
Max: 0.00;
Min: -3.27;
Avg: -0.48. 		hook_pre_linear:
Max: 2.64;
Min: 0.00;
Avg: 0.83. 		hook_pre_linear:
Max: 0.00;
Min: -7.24;
Avg: -1.20.
hook_pre:
Max: 4.10;
Min: 0.00;
Avg: 0.95. 		hook_pre:
Max: 4.13;
Min: 0.00;
Avg: 0.66. 		hook_pre:
Max: 0.00;
Min: -2.32;
Avg: -0.24. 		hook_pre:
Max: 0.00;
Min: -5.43;
Avg: -0.50.
swish:
Max: 4.04;
Min: 0.00;
Avg: 0.71. 		swish:
Max: 4.06;
Min: 0.00;
Avg: 0.46. 		swish:
Max: 0.00;
Min: -0.28;
Avg: -0.09. 		swish:
Max: 0.00;
Min: -0.28;
Avg: -0.15.

Prototypical activations for case gate+_in+

Extreme hook_post activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Extreme hook_pre_linear activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Extreme hook_pre activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Prototypical activations for case gate+_in-

Extreme hook_post activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Extreme hook_pre_linear activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Extreme hook_pre activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Prototypical activations for case gate-_in+

Extreme hook_post activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Extreme hook_pre_linear activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Extreme hook_pre activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Extreme swish activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Prototypical activations for case gate-_in-

Extreme hook_post activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Extreme hook_pre_linear activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Extreme hook_pre activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Extreme swish activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15